Technical and Design Challenges in Multimodal, Situated Human-Robot Dialogue

Allison Sauppé, Bilge Mutlu Department of Computer Sciences University of Wisconsin–Madison 1210 West Dayton Street Madison, WI 53706 USA [email protected], [email protected] Abstract As robotic products become more ubiquitous in society, fulfilling such roles as teachers and assembly-line workers, they will need to be capable of conversing with their users using spoken and nonverbal language. In this position paper, we discuss three challenges that researchers and designers face in creating interfaces for multimodal, situated human-robot dialogue: language synthesis, multimodal input, and task modeling. For each challenge, we present an envisioned scenario of use, discuss prior work in the area, and highlight currently open questions.